Fine-Grained Activity Recognition for Assembly Videos

In this letter we address the task of recognizing assembly actions as a structure (e.g. piece furniture or toy block tower) is built up from set primitive objects. Recognizing full range requires perception at level spatial detail that has not been attempted in action recognition literature to date. We extend fine-grained activity setting its generality by unifying and kinematic structures within single framework. use framework develop general method for observation sequences, along with features take advantage assembly's special structure. Finally, evaluate our empirically on two application-driven data sources: 1) An IKEA furniture-assembly dataset, 2) A block-building dataset. On first, system recognizes an average framewise accuracy 70% normalized edit distance 10%. second, which geometric reasoning distinguish between assemblies, attains 23%-a relative improvement 69% over prior work.